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1J  ABSTRACT 


Variables  used  in  Volume  I,  Study  Results,  which  are  of  the  type 
heavily  influenced  by  management  philosophy,  available  resources, 
etc.,  are  investigated  for  their  effect  on  problem  identification 
test  costs.  Cost  optimizations  u.'  bench  test  versus  flight  test 
were  performed  at  126  data  points.  In  particular,  the  variables 
studied  are:  La}  MTBR  of f-the-board ,  (b)  corrective  action 
efficiency  (c)  test  effectiveness,  and  (d)  test  operating  time/ 
calendar  time.  MTBR  of f-thc-board  and  corrective  action  efficiency 
are  identified  as  the  variables  which  can  heavily  influence  the 
problem  identification  test  costs. 
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DEPARTMENT  OF  THE  ARMY 
U.  S  ARMY  APR  MOBILITY  RESEARCH  A  DEVELOPMENT  LABORATORY 
EUSTIS  DIRECTORATE 
FORT  EUSTIS.  VIRGINIA  23604 


This  report,  Volume  III  of  a  three-volume  report,  was  prepared  by  the 
Boeing-Ver tol  Division  under  the  terms  of  Contract  DAAJ02-70-C-0039 , 
Amendment  POOOL.  It  presents  the  results  of  an  effort  to  explore  the 
relationship  between  test  costs  and  quantitative  reliability  require¬ 
ments,  as  presented  in  Volume  I. 

It  also  examines  the  sensitivity  of  the  t es t -cos t /re  1 labi 1 i ty  relation¬ 
ship  to  those  variables  whose  specific  values  were  selected  through 
engineering  judgments  in  Volume  I. 

Volume  I  gives  the  results  of  a  study  to  establish  the  relationship 
between  various  reliability  demonstration  objectives  and  the  test 
requirements  (type,  hours,  components  required,  cost,  etc.)  necessary  to 
achieve  those  objectives. 

Volume  II  gives  the  results  of  an  effort  to  define  and  discuss  selected 
terminology,  assumptions,  and  variables  used  in  Volume  I.  It  was 
prepared  to  give  the  nonstatistician  a  better  understanding  of  the 
interrelationships  of  the  many  parameters  associated  with  reliability 
demonstration  objectives  and  their  test  requirements,  as  addressed  in 
Volumes  I  and  III. 

The  principal  purposes  of  Volume  III  are: 

a.  To  allow  the  Government  to  evaluate  the  tolerance  of  the  cost 
outputs  to  the  input  assumptions. 

b.  To  identify  areas  that  can  profoundly  influence  the  test-cost/ 
^reliability  relationship,  thus  "flagging"  them  for  a  high  degree  of 

consideration  and  control  by  the  Government  and  its  contractors. 

This  report  is  offered  in  the  continued  interest  of  developing  a  better 
understanding  of  the  relationship  between  helicopter  reliability 
objectives  and  development  test  requirements.  Although  the  point 
estimates  of  time  and  dollars  contained  in  the  report  may  not  apply  to 
all  programs,  the  trends  presented  are  considered  reasonable  and  should 
be  very  useful  in  helicopter  development  program  planning. 

The  technical  monitor  for  this  contract  was  Mr.  Thomas  F..  Condon  of  the 
Reliability  and  Maintainability  Division  of  this  Directorate. 
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SUMMARY 


Variables  used  in  Volume  I,  Study  Results,  which  are  of  the 
type  heavily  influenced  by  management  philosophy,  available  re¬ 
sources,  etc.,  are  investigated  for  their  effect  on  problem 
identification  test  costs.  Cost  optimizations  of  bench  test 
versus  flight  test  were  performed  at  126  data  points.  The 
variables  studied  are: 

a.  MTBR  Of f-the-Board 

b.  Corrective  Action  Efficiency 

c.  Test  Effectiveness 

d.  Test  Operating  Time/Calendar  Time 

MTBR  of f-the-board  and  corrective  action  efficiency  are  identi¬ 
fied  as  variables  which  can  heavily  influence  the  problem 
identification  test  costs. 
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FOREWORD 


This  report  covers  a  study  conducted  under  Contract  DAAJ02-70- 
C-0039,  Amendment  P0001,  (DA  Task  1F162203A14301) ,  to  establish 
the  sensitivity  of  the  reliability  requirement/development  test 
program  cost  relationship  to  changes  in  certain  judgementally 
selected  variables.  This  study  identifies  some  of  the  factors 
that  require  control  in  order  to  obtain  maximum  effectiveness 
from  the  reliability  test  dollar. 

USAAMRDL  technical  direction  was  provided  by  Mr.  Thomas  L. 

House  and  Mr.  Thomas  E.  Condon. 

The  principal  Boeing  Company,  Vertol  Division  investigator  was 
Mr.  Robert  H.  Jines  of  the  Product  Assurance  Methodology  and 
Data  Control  Unit,  assisted  by  Mr.  Kirk  G.  Rummel.  Program 
management  and  technical  direction  were  provided  by  Mr.  G.  W. 
Windolph,  Manager,  Product  Assurance  Technical  Staff,  and  Mr. 

R.  B.  Aronson,  Unit  Chief,  Product  Assurance  Methodology  &  Data 
Control  Unit. 
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DEFINITION  OF  TERMS 


In  this  volume,  this  terminology  is  used  to 
categorize : 

MTBR-Of f-the-Board 
Test  Effectiveness 
Corrective  Action  Efficiency 
Operating  Time/Calendar  Time  Ratio 

In  this  Volume,  CASE  is  used  to  describe  the  in¬ 
dividual  situations  that  were  evaluated  for  a 
variable.  For  example,  the  following  CASES  were 
evaluated  under  the  MTBR-Of f-the-Board  variable: 

High-Frequency  Failure  Modes  Removed 
Low-Frequency  Failure  Modes  Removed 
25%  of  Baseline  Failure  Modes  Present 
200%  of  Baseline  Failure  Modes  Present 

The  term  VALUE  is  used  to  indicate  the  cost  ob¬ 
tained  for  each  individual  case. 


BASELINE 


The  term  BASELINE  refers  to  the  particular  case 
evaluated  in  Volume  I  or  to  the  numerical  value 
presented  for  that  case. 


1.0  INTRODUCTION 


Maintenance  requirements  for  helicopters  have  always  been 
greater  than  for  comparable  fixed-wing  aircraft.  Helicopter 
components  are  relatively  complex,  particularly  in  the  drive, 
rotor,  and  upper  controls  systems.  Since  failures  in  these 
systems  have  a  potential  for  causing  accidents,  maximum  reli¬ 
ability  becomes  paramount.  Further,  high  acquisition  and 
repair  costs  make  maximum  reliability  a  still  greater  neces¬ 
sity.  Both  the  user  and  the  contractor  see  the  need  for  im¬ 
proved  reliability;  but,  until  now,  the  development  cost 
impact  has  been  unknown,  and  reliability  growth  by  product 
improvement  has  been  slow  and  costly. 

Hardware  reliability  is  produced  by  two  life-cycle  activities, 
as  illustrated  by  Figure  3  of  Volume  I ,  "Growth  of  Component 
Reliability  Through  Design  and  Test".  The  first,  the  design 
process,  results  in  the  MTBR  of f -the-board  of  the  hardware. 
Design  analysis  activity  is  probably  the  most  cost-effective 
process  for  obtaining  high  reliability,  since  potential  prob¬ 
lems  are  identified  and  corrected  before  hardware  is  cut.  The 
technology  for  achieving  the  required  reliability  by  this 
process  requires  further  development,  however.  The  second 
activity  is  the  process  of  discovering  and  eliminating  prob¬ 
lems  in  the  hardware  by  operating  it,  either  in  special  test 
or  in  actual  service  use. 

Most  programs  use  the  design  and  operation  activities  in  com¬ 
bination  to  achieve  reliability.  This  approach  will  continue, 
with  the  emphasis  shifting  more  to  problem  identification 
through  design  analysis,  as  the  technology  for  this  is  im¬ 
proved.  With  current  technology,  however,  extensive  problem 
identification  testing  must  be  used  to  produce  the  required 
reliability  level. 

Determining  appropriate  levels  of  Problem  Identification  Test¬ 
ing  to  develop  components  that  satisfy  reliability  require¬ 
ments  necessitates  consideration  of  development  test  costs, 
schedules,  mixes  of  techniques,  demonstration  criteria,  and 
hardware  characteristics.  Methods  for  putting  these  elements 
into  proper  perspective  were  presented  in  Volume  I.  Since  the 
true  return  from  the  problem  identification  test  funds  invested 
is  the  resulting  actual  MTBR  of  the  components,  it  is  highly 
desirable  to  understand  the  sensitivity  of  Problem  Identifica¬ 
tion  Test  costs  to  changes  in  the  input  variables.  The  cost 
impact  of  all  rigorously  quantifiable  variables  was  explored  in 
Volume  I.  Other  variables,  heavily  influenced  by  user/con¬ 
tractor  management  philosophy,  and  hence  less  rigorously 
quantifiable,  were  evaluated  for  single  values  only  in  Volume  I. 
This  sensitivity  study.  Volume  III,  evaluates  the  impact  upon 
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problem  identification  test  costs  of  several  of  these  manage¬ 
ment  philosophy-influenced  variables,  and  identifies  those 
that  have  great  cost  leverage  so  that  they  can  receive  the 
brunt  of  management  attention.  Presented  herein  are  the  ap¬ 
proach,  observations,  and  conclusions  pertinent  to  the  sensi¬ 
tivity  analysis  of  the  following  variables: 

a.  MTBR  Off-the-Board 

b.  Corrective  Action  Efficiency 

c.  Fraction  of  Test  Restraints  Removed 

d.  Test  Operating  Time/Calendar  Time 

The  range  of  values  investigated  for  each  variable  was  based 
upon  a  judgement  of  the  current  engineering  and  management 
technology. 

Figure  1  illustrates  the  interrelationships  among  the  elements 
affecting  reliability  test  program  costs;  it  also  illustrates 
the  points  in  the  flow  process  where  the  four  variables  under 
investigation  exert  their  respective  influence. 


TEST  EFFECTIVENESS 


©TEST  OPERATING  TIME/ 
CALENDAR  TIME 


Figure  1.  Flow  Chart  Showing  Entry  of  the  Study  Variables 
Into  the  Test  Program  Cost  Process. 
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2.0  INDIVIDUAL  VARIABLES  UNDER  INVESTIGATION 


Each  of  the  four  variables  investigated  was  considered  to  have 
great  potential  for  impacting  cost. 

2.1  MTBR  OFF -THE -BOARD 

Design  and  analysis  activities  are  those  activities  performed 
during  the  design  phase  (prior  to  building  and  testing  hard¬ 
ware)  to  improve  the  reliability  of  the  hardware.  They 
include : 

•  Development  of  Criteria 

•  Goal  Apportionment 

•  Development  of  Specifications 

•  Reliability  Prediction 

•  Failure  Mode  and  Effect  Analysis 

•  Design  Review 

Also  included  is  a  series  of  specialized  tests,  called  design 
development  tests,  that  occur  early  in  the  design  phase. 

MTBR  of f-the-board  is  the  reliability  level  produced  by  these 
design  and  analysis  activities  prior  to  application  of  any 
reliability  problem  identification  testing. 

MTBR  of f-the-board  is  predicted  using  historical  data  from 
similar  hardware  by  analyzing  the  design  to  identify  suspected 
failure  modes  and  their  frequencies.  The  predictions  are  ap¬ 
propriate  only  for  an  aircraft  of  specific  size  and  configura¬ 
tion.  Engineering  judgement,  which  requires  a  subjective 
consideration  of  the  contractor's  state-of-the-art  ability  to 
make  designs  problem  free,  is  used  in  the  prediction  process. 
This  state  of  the  art  varies  from  component  to  component,  from 
year  to  year,  and  from  contractor  to  contractor.  The  MTBR 
of f-the-board  predictions  presented  in  Volume  I  reflect  the 
individual  component  design  (size,  weight,  configuration, 
loads,  etc.)  and  the  specific  time  at  which  the  design  would 
occur.  The  predictions  reflect  the  experience  of  one  contrac¬ 
tor,  with  a  specific  level  of  effort  expended  on  initial  design. 

In  today's  climate  of  contractual  reliability  requirements  the 
MTBR  predictions  presented  in  Volume  I  are  realistic.  The  fol¬ 
lowing  specific  cases  are  investigated  in  the  sensitivity 
study  reported  in  Volume  III. 
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Case 


Degree  of  Realism 


25%  of  baseline  failure 
modes  present 

Volume  I  baseline 

200%  of  baseline  failure 
modes  present 

High-frequency  failure 
modes  (1000-hour  MTBR  or 
less)  not  present 

Low-frequency  failure 
modes  (10,000-hour  MTBR 
or  greater)  not  present 


Optimistic,  improbable 


Expected  value 

Indicative  of  poor  per¬ 
formance 

Excessive  emphasis  on 
frequency* 


Artificial,  unrealistic, 
improbable . 


2.2  CORRECTIVE  ACTION  EFFICIENCY  -  (Number  of  Observations 
Required  During  Problem  Identification  Testing  To 
Postulate  Corrective  Action) 


In  Volume  I,  problem  identification  test  costs  were  established 
by  devising  and  costing  test  programs  of  sufficient  duration  to 
produce  the  required  MTBR  considered  necessary  to  enter 
demonstration . 


The  MTBR  of f-the-board  had  been  predicted  for  each  failure 
mode  anticipated  in  the  component.  Test  durations  for  the  test 
techniques  being  used  were  calculated  based  on  these  failure 
modes  and  their  frequencies  (reciprocal  of  MTBR  of f-the-board) 
and  consideration  of  the  required  MTBR  for  the  component. 

Test  conditions  (load  magnitude,  cyclic  rate)  and  test  effec¬ 
tiveness  for  the  test  technique  were  considered. 

It  was  postulated  that  a  test  duration  equal  to  the  MTBR  off- 
the-board  of  a  failure  mode  would  produce  one  occurrence 
(observation)  of  the  failure  mode.  In  Volume  I  it  was  postu¬ 
lated  that  in  a  test  duration  of  twice  the  MTBR  of f-the-board 
of  a  particular  failure  mode,  the  problem  would  be  identified, 
understood,  and  adequate  design  corrective  action  devised,  so 
that  the  failure  mode  would  never  occur  again  (termed  "Two 
Observations  To  Fix").  In  this  manner,  test  programs  were 
sized  to  detect  and  correct  sufficient  failure  modes  so  that 
the  net  effect  of  the  anticipated  remaining  modes  equaled  the 
required  MTBR  of  the  component. 


♦This  case  would  result  from  ranking  of  problems  in  order  of 
frequency  of  occurrence  instead  of  assessing  problem  con¬ 
sequence  and  the  cost  effectiveness  of  corrective  action. 
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The  two-observations -to-fix  philosophy  postulated  in  Volume  I 
is  optimistic,  even  where  a  contractual  numerical  reliability 
demonstration  requirement  is  imposed.  Three  or  even  four  ob¬ 
servations  to  postulate  development  of  corrective  action  is 
more  realistic.  For  the  study  reported  herein,  the  following 
specific  cases  were  studied: 

Case 


2  Observations  to  fix 
(Volume  I  baseline) 

4  Observations  to  fix 

8  Observations  to  fix 


2.3  TEST  EFFECTIVENESS 

Effectiveness  is  defined  as  the  ability  of  a  test  technique  to 
detect  a  potential  field  problem.  Correction  of  problems 
after  they  are  detected  does  not  differ  significantly  among 
test  techniques.  Therefore,  the  main  concern  is  the  detection 
ability  of  each  test  technique. 

In  support  of  the  evaluation  reported  in  Volume  I,  CH-47  test 
and  service  experience  was  reviewed.  A  large  percentage  (74 
percent)  of  CH-47  problems  were  detected  by  the  total  test 
program,  but  no  individual  test  technique  detected  all  the 
problems.  Furthermore,  the  full  detection  potential  of  each 
test  technique  was  not  manifested  in  the  test  results.  Many 
problems  were  not  detected  during  certain  tests  for  reasons 
which  were  completely  unrelated  to  the  actual  test  techniques 
used.  These  reasons  were  termed  artificial  restraints.  The 
term  inherent  restraints  was  used  for  those  test  technique 
limitations  which  stem  from  the  specific  load,  speeds,  config¬ 
urations,  or  climatic  environments  (dust,  humidity,  tempera¬ 
ture,  etc.)  of  each  test  technique. 

The  artificial  restraints  observed  in  the  CH-47  test  program 
were  classified  as  follows: 

Configuration 

Many  problems  which  first  appeared  in  early  tests  were  com¬ 
pletely  corrected.  Later  test  specimens  incorporated  these 
design  changes  and  therefore  did  not  fail  during  subsequent 
tests  using  other  techniques. 


Degree  of  Realism 
Optimistic 


Realistic 

Indicative  of  unwillingness 
to  acknowledge  problems  or 
inefficiency  in  correcting 
them 


In  other  instances,  a  failure  mode  caused  by  a  manufacturing 
or  material  error  happened  to  appear  in  a  specimen  that  was 
being  tested  by  a  given  test  technique.  Other  test  techniques 
could  have  detected  the  failure  mode  if  the  necessary  con¬ 
ditions  were  present  in  the  specimen. 

Maintenance 

Maintenance  damage  caused  many  failure  modes.  Often,  these 
failure  modes  were  not  detected  during  test  because  of  the  un¬ 
real  maintenance  environment.  The  manner  in  which  the  hard¬ 
ware  was  installed  in  the  test  fixture  also  precluded  the  ap¬ 
pearance  of  certain  maintenance-induced  problems. 

Test  Acceptance  Criteria 

A  significant  number  of  problems  actually  occurred  during  tests 
but  were  not  recorded,  reported,  or  corrected  because  the 
criteria  under  which  the  test  was  operating  did  not  require 
such  recognition.  Failure  modes  which  involved  a  measure  of 
degree  (e.g.,  wear,  fretting,  leakage)  were  particularly  sus¬ 
ceptible  to  being  overlooked.  Emphasis  was  placed  upon  those 
modes  which  caused  the  component  to  fail  to  operate. 

Test  Procedures 


In  certain  instances,  the  design  contained  features  which  were 
not  used  during  testing.  In-service  use  of  these  features  re¬ 
sulted  in  subsequent  failures.  For  example,  a  quick-disconnect 
feature  was  incorporated  into  the  CH-47  lag  damper/rotor  head 
assembly.  Use  of  this  feature  in  the  field  soon  resulted  in 
the  discovery  of  an  understrength  sheet  metal  bracket,  the  re¬ 
pair  of  which  required  rotor  head  removal.  Exercising  this 
quick-disconnect  feature  during  whirl  tower  testing  would  have 
brought  out  the  full  problem  detection  potential  of  the  whirl 
tower  test. 

The  above  artificial  restraints  prevented  many  problems  from 
being  detected  by  specific  test  techniques.  They  did  not,  how¬ 
ever,  detract  from  the  problem-detection  potential  of  a  given 
test  technique.  Also,  a  test  technique  might  not  have  been 
used  long  enough  to  detect  the  problem.  If  a  failure  mode  had 
MTBR  in  excess  of  the  test  duration,  it  would  have  a  low  prob¬ 
ability  of  appearance  despite  the  fact  that  there  was  no 
inherent  restraint. 

For  these  reasons,  a  comparison  of  test  and  field  identified 
problems  must  acknowledge  four  categories: 

•  Problems  that  were  actually  detected  during  test  tech¬ 
nique  application 
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•  Problems  that  could  have  been  detected  except  for  the 
presence  of  artificial  restraints 

•  Problems  that  could  have  been  detected  if  the  test  tech¬ 
nique  had  been  used  long  enough 

•  Problems  that  could  not  be  detected  because  of  inherent 
restraints 


The  full  detection  potential  of  any  test  technique  is  repre¬ 
sented  by  the  sum  of  the  first  three  categories. 

For  the  Volume  I  study,  it  was  assumed  that  all  artificial  re¬ 
straints  were  removed.  This  is  considered  to  be  a  reasonable 
assumption  given  a  contractual  reliability  demonstration  re¬ 
quirement.  The  following  specific  cases  were  investigated  in 
this  study: 


Case 


All  (artificial  and  in¬ 
herent)  restraints  removed 


Volume  I  baseline  (all 
artificial  restraints 
removed) 

Maintenance  and  test 
acceptance  criteria  arti¬ 
ficial  restraints  not 
removed  (test  procedure 
and  configuration  artific 
restraints  removed) 

Table  I  contains  the  test  effectiveness  percentages  used  for 
the  Sensitivity  Study. 

2.4  TEST  TECHNIQUE  OPERATING  TIME/CALENDAR  TIME  RELATIONSHIP 

Schedules  for  each  test  technique  are  defined  by  two  distinct 
characteristics.  The  first  is  the  lead  time  necessary  from 
the  go-ahead  decision  to  the  point  where  the  test  fixture  is 
available  for  operation.  The  second  is  the  quantity  of  test 
hours  that  can  be  accumulated  in  a  given  calendar  period.  This 
has  been  termed  the  operational  rate.  Both  factors  are  ex¬ 
tremely  important  in  determining  the  costs  of  test  programs, 
since  they  determine  the  number  of  test  rigs  required  to  ful¬ 
fill  a  given  test  duration  in  a  given  calendar  period. 


Degree  of  Realism 

Reflects  ideal  case  where 
test  fixture  has  flight 
test  effectiveness 

Realistic 


Reflects  unrealistic 
attitude  for  the  contrac¬ 
tual  environment 
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Component 

Test 

Fixture 

With  In¬ 
herent  and 
Artificial 
Restraints 

Artificial 

Restraints 

Removed 

No 

Restraints 

Main  Xmsn 

Closed 

Loop 

80% 

92% 

100% 

Main  Hub 

Whirl 

Tower 

85 

85 

100 

Main 

Controls 

Whirl 

Tower 

75 

85 

100 

Main 

Controls 

Bench 

75 

85 

100 

Main 

Blades 

Whirl 

Tower 

65 

65 

100 

Intermediate 

Gearbox 

Tail 

Rotor 

Stand 

67 

90 

100 

Tail  Rotor 
Gearbox 

Tail 

Rotor 

Stand 

65 

85 

100 

Tail  Rotor 
Hub 

Tail 

Rotor 

Stand 

85 

85 

100 

Tail  Rotor 
Blade 

Tail 

Rotor 

Stand 

65 

65 

100 

Tail  Rotor 
Shaft 


Tail 

Rotor 

Stand 


90 


90 


100 


The  schedule  characteristics  presented  in  Volume  I  were 
derived  through  review  of  data  on  the  CH-47.  Subsequently, 
adjustments  were  made  to  reflect  certain  assumptions: 

1.  All  tests  operated  for  3  shifts  each  day,  7  days  a  week. 

2.  No  interruptions  in  developmental  tests  permitted  for 
production  usage  of  the  test  facility. 

3.  Additional  specimens  available  to  immediately  replace 
test  specimens  which  have  failed  or  require  removal. 

4.  Test  stand  reliability  representative  of  mature 
equipment . 

5.  Lead  times  to  procure  fixtures  assume  a  degree  of  pre¬ 
implementation  traditional  in  aircraft  programs,  i.e., 
basic  conceptual  and  sizing  efforts  completed,  but  detail 
design  must  await  contract  go-ahead. 

The  following  helicopter  'A'  test  technique  operational  rates 
were  used  in  Volume  I: 


Test  Technique 

Operational  Rate 
(hours/month) 

Controls  bench,  back-to-back 

500 

Controls  bench,  single  specimen 

500 

Tiedown 

165 

Dynamic  systems  test 

200 

Whirl  tower 

350 

Transmission  closed  loop 

400 

Tail  rotor  whirl  tower 

400 

Alaska  climatic 

20 

Yuma  climatic 

20 

Flight 

70 

The  above  rates  present  an  upper  limit  of  contractor  efficiency. 
This  sensitivity  study,  accordingly,  investigates  the  following 
cases : 


Case 


Degree  of  Realism 


Volume  I  Baseline 


0.5  x  Baseline 


Optimistic,  represents  an 
achievable  upper  limit 

Realistic,  achievable  with 
reasonable  effort 
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case 


Degree  of  Realism 


0.25  x  Baseline  Not  representative  of  a 

reasonable  effort,  in  a 
contractual  reliability 
demonstration  environment 

Table  II  shows  the  limiting  run  time  per  rig  used  in  the  study 


TABLE  II.  TEST  RIG  RUN  TIMES 


Test  Rig 

Program 

Limiting 

j  Run  Time  Pe: 

Ln  p 

(Years) 

Study 

0.5  x  Study 

Xmsn 

3 

7,200 

3,600 

Closed 

4 

12,000 

6,000 

Loop 

6 

21,600 

10,800 

Whirl 

3 

5,600 

2,800 

Tower 

4 

9,800 

4,900 

6 

18,200 

9,100 

Tail 

3 

8,000 

4,000 

Rotor 

4 

12,800 

6,400 

Stand 

6 

22,400 

11,200 

Control 

3 

15,000 

7,500 

Bench 

4 

21,000 

10,500 

6 

33,000 

16,500 

1 , 80( 
3 , 00( 

5 . 4  0  ( 

1 , 40( 

2 . 4  5  ( 
4 , 55( 

2,000 

3,200 

5,600 

3,750 

5.250 

8.250 


3.0  STUDY  PROCEDURES  AND  GROUND  RULES 


3.1  PROCEDURES 

The  calculation  techniques  for  the  basic  study,  Volume  I,  with 
the  following  modifications,  define  the  approach  to  the  sensi¬ 
tivity  study: 

•  Cost,  schedule,  failure  modes  and  failure  rate  distribu¬ 
tions,  and  other  data  are  the  same  as  used  for  Volume  I. 
This  includes  data  presented  in  Tables  XV  &  XVI,  and 
Appendices  II  &  IV  of  Volume  I.  No  new  data  was  gener¬ 
ated  for  the  sensitivity  study. 

•  All  values  used  were  held  constant,  with  the  exception  of 
the  value  of  the  variable  being  studied.  In  the  "com¬ 
bined  worst  case"  analysis, all  investigated  variables 
were  altered  simultaneously. 

•  A  study  was  performed  for  a  bench  plus  flight  type  pro¬ 
gram,  and  a  flight  test  versus  bench  test  cost  optimiza¬ 
tion  analysis  was  performed  for  each  case  of  each  vari¬ 
able  under  study.  (This  procedure  resulted  in  126  cost 
data  points  being  calculated.  A  sample  cost  optimization 
plot  is  shown  in  Figure  2,  which  illustrates  the  pro¬ 
cedure  followed.) 

•  Each  cost  data  point  was  divided  by  the  associated  base¬ 
line  cost  from  Volume  I  to  normalize  the  data  for  com¬ 
parison  purposes.  Baseline  costs  are  tabulated  in 
Table  V,  page  38. 

•  The  resulting  cost  normalizations  were  then  used  to  eval¬ 
uate  problem  identification  test  cost  trends  vs.  program 
elapsed  time.  The  calculation  involved  averaging  appro¬ 
priate  600- ,  3000- ,  &  5200-hour  MTBR  costs  for  a  3-  or 
6-year  program  and  dividing  the  result  by  a  similarly 
averaged  value  for  a  4-year  program.  Table  VI  is  a  tab¬ 
ulation  of  the  resulting  ratios  for  each  study  case. 

3.2  GROUND  RULES 

The  following  ground  rules  were  applied  to  the  study: 

•  3-,  4-,  and  6-year  development  program  elapsed  time  data 
points  were  calculated. 

•  Only  the  600- ,  3000-  and  5200-hour  required  MTBR  data 
points  were  calculated. 


COMBINED  BENCH  &  FLIGHT  TEST  COSTS  ($  MILLIONS) 


TO  PRODUCE  REQUIRED 
MTBR)  I  I  I 


FLIGHT  TEST  DURATION  (1000  HOURS) 


Figure  2.  Example  of  the  Optimization  Process 

(Decreasing  Flight  Test  Duration  Causes 
Increased  Bench  Test  Cost) . 
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•  Problem  identification  test  durations  for  each  test  rig 
are  derived  for  the  component  which  paced  that  rig  in 
the  basic  study  (Volume  I)  for  all  study  variables  ex¬ 
cept  "Test  Effectiveness".  (Verification  checks  indi¬ 
cated  that  the  pacing  components  would  not  change.)  In 
the  case  of  Test  Effectiveness,  all  components  were  re¬ 
viewed  to  identify  the  pacing  component,  since  individual 
artificial  restraints  operated  on  individual  component 
failure  modes. 

•  Only  problem  identification  test  costs  (Type  II)  were 
included  in  the  sensitivity  analysis.  (Demonstration 
costs  and  Type  I  costs  were  not  considered.) 

•  Only  Helicopter  'A*  analyses  were  performed. 

•  Component  bench  tests  were  supplemented  by  Type  I  and  II 
flight  test.  1500  hours  of  "free"  Type  I  flight  test 
were  applied  to  each  program. 

•  The  number  of  aircraft  available  for  both  Type  I  and 
Type  II  testing  was  considered  to  be  limited  only  by  the 
aircraft  delivery  schedule.  The  delivery  rate  was  as¬ 
sumed  to  be  three  aircraft  per  month  for  this  study. 

Table  III  shows  the  maximum  total  flight  test  hours  pos¬ 
sible  for  3-,  4-,  and  6-year  problem  identification  pro¬ 
grams.  Lead  time  for  delivery  of  the  first  aircraft  was 
2  years  in  every  case.  The  minimum  number  of  flight 
hours  considered  adequate  on  any  individual  aircraft  was 
140.  Because  the  number  of  aircraft  produced  is  inde¬ 
pendent  on  the  length  of  production,  1-  and  2-year 
delivery  programs  are  shown. 


TABLE  III. 

LIMITING  VALUES  FOR  FLIGHT  TESTING 

Program 

Length 

(Years) 

Number  of 

Aircraft 

Total  Flight  1 
Type  I  and  Ty 

tours  Possible, 
fpe  II  Flight 

1-Year 

Delivery 

2 -Year 
Delivery 

l-Year 

Delivery 

2-Year 

Delivery 

3 

30 

N/A 

12,000 

N/A 

4 

30 

66 

37,200 

58,620 

6 

30 

66 

87,600 

114,060 
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If  program  duration  permits  a  2-year  delivery  schedule  for  the 
flight  test  aircraft,  sufficient  flight  hours  for  the  test 
programs  can  always  be  developed.  For  the  3-year  program  dur¬ 
ation,  the  2-year  lead  time  requirement  prevents  a  2-year 
delivery  schedule.  In  these  cases,  the  aircraft  delivery 
period  was  restricted  to  1  year.  In  many  of  these  cases,  this 
provided  sufficient  flight  test  hours  for  program  optimization. 
However,  in  a  few  cases,  as  in  Figures  6,  10,  13,  14,  and  15, 
this  was  not  so,  and  less  efficient  test  techniques  had  to  be 
used  in  order  to  reach  the  required  MTBR  levels. 

In  a  typical  test  program,  it  is  desirable  to  get  high  test 
times  on  each  aircraft.  This  approach  causes  the  program  to 
strive  for  production  of  a  minimum  number  of  aircraft  with 
delivery  as  early  as  possible.  However,  the  question  "What 
happens  to  the  production  line  in  the  period  between  the  end 
of  delivery  of  problem  identification  aircraft  and  the  start 
of  production  delivery?"  must  eventually  be  considered. 


4.0  DETAILED  RESULTS  AND  DISCUSSION 
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4.1  MTBR  OFF-THE-BOARD 


Results 


The  test  programs  resulting  from  the  flight  test  versus  bench 
test  cost  optimization  procedure  application  for  the  various 
MTBR  of f-the-board  variations  are  shown  in  Appendix  I.  The 
number  of  test  rigs,  test  rig  acquisition  costs,  test  rig  run 
costs,  number  of  flight  test  hours  in  program,  flight  test 
costs,  total  run  costs  (test  rig  plus  flight  test),  and  total 
program  costs  (acquisition  plus  run  costs)  are  listed. 

The  3-,  4-,  and  6-year  development  program  "Problem  Identifi¬ 
cation  Test  Costs"  vs.  "Required  MTBR  Achieved"  plots  are  con¬ 
tained  in  Figures  16,  17,  and  18,  in  Appendix  II. 

Figures  3,  4,  and  5  plot  the  ratio  of  test  cost/baseline  test 
costs  versus  required  MTBR  achieved  for  the  variations  in 
MTBR  of f-the-board ,  for  3-,  4-,  and  6-year  programs.  In  each 
figure,  Case  “A"  represents  the  baseline  case  evaluated  in 
Volume  I . 

Table  V,  page  38,  lists  the  absolute  values  of  problem  identi¬ 
fication  test  program  costs  and  test  cost/baseline  test  cost 
ratios  for  each  of  the  cases  evaluated,  as  well  as  the  baseline 
case  absolute  values  from  Volume  I. 

Discussion 


MTBR  of  f-the-board ,  in  general,  was  found  to  be  a  powerful 
driving  force  upon  the  numerical  reliability  requirement/test 
program  cost  relationship.  The  following  discussions  pertain 
to  the  individual  casec'  investigated. 

Elimination  of  High-Frequency  Failure  Modes 


Of  all  the  MTBR  of  f-the-board  cases  investigated,  elimination 
of  those  failure  modes  with  a  high  frequency  of  occurrence  had 
the  least  impact  on  cost.  The  problem  identification  test  dur¬ 
ations  required  to  produce  the  typical  level  of  "required  MTBR 
achieved"  are  such  that  all  of  the  high-frequency  failure 
modes  and  many  relatively  low-frequency  failure  modes  must  be 
identified  and  corrected.  Detection  of  the  low-frequency  modes 
requires  problem  identification  test  durations  in  excess  of 
those  required  to  detect  the  high-frequency  modes.  Application 
of  1500  hours  of  "free"  (Type  I)  flight  test  to  the  problem 
identification  test  program  further  desensitizes  the  effect  of 
high-frequency  mode  elimination. 
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ST  COST/BASELINE  TEST  COST 


CASE:  | 

_  A  =  BASELINE  I  I 

B  =  HIGH-FREQUENCY  MODES  ELIMINATED 
C  =  LOW-FREQUENCY  MODES  ELIMINATED 

—  D  =  NO.  OF  FAILURE  MODES  X  1/4  J - 

E  =  NO.  OF  FAILURE  MODES  X  2  | 


H  1 


REQUIRED  MTBR  ACHIEVED  (  1000  HRS) 

Figure  3.  3-Year  Program  MTBR  Of f -the-Board  Sensitivity  - 
Ratio  of  Study  Value  and  Baseline  Value  Problem 
Icientif ication  Test  Costs. 
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REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

Figure  4.  4-Year  Program  MTBR  Of f-the-Board  Sensitivity  - 
Ratio  of  Study  Value  and  Baseline  Value  Prob¬ 
lem  Identification  Test  Costs. 
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TEST  COST/BASELINE 


For  Example: 


For  the  "two  observations  to  postulate  corrective  action" 
philosophy,  all  failure  modes  of  1000  hours  or  less  are 
assumed  to  be  identified  and  corrected  with  2000  hours  of 
problem  identification  testing.  Seventy-five  percent  of 
these  failure  modes  are  accounted  for  by  the  "free"  Type  I 
flight  testing. 

Results  of  sensitivity  analysis  indicated  no  change  in  test 
costs  for  any  of  the  values  calculated,  through  elimination  of 
all  modes  with  a  1000-hour  MTBR  or  less. 

There  is  a  significant  lesson  to  be  learned  here.  Tradition¬ 
ally,  attention  in  the  reliability  area  has  been  focused  upon 
the  "Top  Ten  (or  more)  Problems"  ranked  by  frequency  of  occur¬ 
rence.  The  sensitivity  study  shows  that  this  emphasis  has  no 
impact  on  test  costs  and,  as  shown  in  Volume  I,  is  inadequate 
to  produce  the  levels  of  reliability  necessary  for  the  next 
generation. 

Elimination  of  Low-Frequency  Failure  Modes 


For  higher  values  of  required  MTBR  achieved  (3000  and  5200 
hours) ,  elimination  of  the  low-frequency  failure  modes  has  a 
profound  effect  on  test  costs. 

No  change  in  test  costs  was  observed,  however,  for  the  600- 
hour  required  MTBR  achieved  value. 

From  a  practical  standpoint,  it  will  be  difficult  to  profit 
from  this  approach  in  an  applied  program.  Emphasis  has  always 
been  placed  upon  elimination  of  problems  that  occur  frequently. 
Infrequently  occurring  failures  traditionally  are  dismissed  as 
isolated  cases  and  are  usually  ignored.  Yet  it  is  this  pool  of 
isolated  failure  modes  that  prevents  high  MTBR ' s  from  being 
achieved.  In  the  next  generation  of  hardware  development, 
considerable  emphasis  must  be  placed  upon  elimination  of  fail¬ 
ure  modes  over  a  wide  range  of  frequencies  in  the  off-the- 
board  design. 

Changes  Affecting  the  Entire  Failure  Mode  Distribution 

Increases  or  decreases  in  the  number  of  failure  modes  across 
the  spectrum  of  frequencies  have  a  significant  influence  upon 
test  costs,  and  represent  conditions  that  can  be  encountered 
in  applied  programs. 

Poor  design  practices  and  little  attention  paid  to  reliability 
in  the  trade-off  process  will  probably  result  in  an  across-the- 
board  increase  in  the  number  of  failure  modes  present  in  the 
design. 
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Similarly,  an  across-the-board  decrease  in  the  number  of  fail¬ 
ure  modes  present  is  likely,  if  careful  attention  is  paid  to 
design  and  those  factors  which  improve  reliability. 

The  sensitivity  study  indicates  a  nearly  direct  relationship 
between  the  percentage  of  change  in  problem  identification  test 
costs  and  the  percentage  of  change  in  the  number  of  failure 
modes  present,  except  in  the  area  of  large  reductions  in  the 
number  of  modes  in  the  relatively  low  required  MTBR  achieved 
range.  This  latter  effect  results  from  the  assumption  that 
1500  hours  of  "free"  (Type  I)  flight  test  is  available,  which 
tends  to  decrease  cost  sensitivity  for  relatively  short  prog- 
grams. 

4.2  CORRECTIVE  ACTION  EFFICIENCY 
Results 


The  test  programs  resulting  from  the  flight  test  versus  bench 
test  cost  optimization  for  each  corrective  action  efficiency 
study  case  are  shown  in  Appendix  I. 

The  3-,  4-,  and  6-year  problem  identification  test  costs  versus 
required  MTBR  achieved  plots  are  contained  in  Figures  19,  20, 
and  21  in  Appendix  II. 

Figures  6,  7,  and  8  contain  plots  of  costs  relative  to  the  as¬ 
sociated  baseline  versus  required  MTBR  achieved  for  3,  4-,  and 
6-year  programs,  respectively.  In  each  figure,  Case  "A"  rep¬ 
resents  the  baseline  case  evaluated  in  Volume  I. 

Absolute  values  of  problem  identification  program  test  cost  and 
the  cost  relative  to  the  associated  baseline  cost  for  each 
corrective  action  efficiency  value  are  summarized  in  Table  V, 
on  page  38.  Table  V  also  lists  the  baseline  case  absolute 
values  from  Volume  I. 

Discussion 


Corrective  action  efficiency  is  a  prime  factor  driving  the 
problem  identification  test  costs.  In  general,  the  percentage 
of  test  cost  increase  is  equal  to  the  percentage  of  increase 
in  the  test  duration  postulated  to  result  in  correction  of  a 
failure  mode.  For  example,  in  most  circumstances,  where 
testing  durations  equal  to  four  times  the  MTBR  are  required  for 
problem  correction,  the  test  costs  are  roughly  twice  the  cost 
of  a  program  where  testing  durations  equal  to  twice  the  MTBR 
are  required.  For  low  values  of  required  MTBR,  the  relation¬ 
ship  does  not  hold,  due  to  the  variations  in  the  effect  of  the 
"free"  Type  I  flight  testing.  (Type  I  testing,  here,  repre¬ 
sents  a  large  portion  of  the  total  testing.) 
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REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

Figure  6.  3-Year  Program  Corrective  Action  Efficiency 
Sensitivity  -  Ratio  of  Study  Value  and  Base¬ 
line  Value  Problem  Identification  Test  Costs. 
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TEST  COST/BASELINE  TEST  COST 


TEST  COST/BASELINE  TEST  COST 
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The  effect  of  the  above  discussion  is  seen  in  Figures  6,  7, 
and  8  as  a  rise  in  the  curves  as  required  MTBR  is  decreased. 

From  a  practical  standpoint,  corrective  action  efficiency  is 
significant  in  an  applied  program.  Willingness  to  acknowledge 
and  correct  problems  is  greatly  influenced  by  contract  require¬ 
ments.  Hence,  these  requirements  have  a  direct  influence  on 
the  cost  of  the  problem  identification  test  program  in  terms 
of  receptiveness  to  problem  acknowledgement. 

4 . 3  TEST  EFFECTIVENESS 


Results 


The  optimized  test  programs  for  the  test  effectiveness  sensi¬ 
tivity  study  are  shown  in  Appendix  I. 

The  3-,  4-,  and  6-year  problem  identification  test  programs 
cost  versus  required  MTBR  achieved  plots  are  contained  in 
Figures  22,  23,  and  24  of  Appendix  II. 

Figure  9  contains  the  plot  of  cost  relative  to  the  associated 
baseline  versus  required  MTBR  achieved  for  the  3-,  4-,  and 
6-year  programs.  Case  "A"  represents  the  baseline  case  eval¬ 
uated  in  Volume  I . 

Problem  identification  test  costs  and  the  cost  relative  to  the 
associated  baseline  costs  for  each  test  effectiveness  study 
are  summarized  in  Table  V,  on  page  38.  Table  V  lists  the  base 
line  case  absolute  values  from  Volume  I. 

Discussion 


The  process  of  optimizing  flight  test  vs.  bench  test  reduces 
the  sensitivity  of  problem  identification  test  costs  to  test 
effectiveness.  The  relative  insensitivity  of  costs  to  this 
variable  are  best  established  by  considering  the  limiting  cases. 

Maximum  costs  would  occur  if  the  bench  test  effectiveness  were 
zero.  Then,  the  test  program  would  become  all  flight  test,  with 
an  associated  cost  of  from  1.5  to  2.0  times  the  baseline  prob¬ 
lem  identification  test  costs.  Minimum  costs  would  occur  if 
there  were  no  inherent  or  artificial  restraints  imposed  upon 
bench  test  (100%  test  effectiveness) .  In  this  case,  the  test 
program  would  be  all  bench  test  except  for  the  "free"  type  I 
flight  test.  Here,  minimum  costs  range  from  0.5  to  1.0  times 
the  baseline  problem  identification  test  costs. 

For  the  realistic  case  where  maintenance  and  test  acceptance 
criteria  artificial  restraints  are  not  eliminated,  the  problem 
identification  test  costs  are  approximately  1.3  times  the 
baseline  value. 
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TEST  COSTS/BASELINE  TEST  COSTS 


,4,6 


BASELINE  “1 - ' - ' - ' 

MAINTENANCE  AND  TEST  ACCEPTANC: 
CRITERIA  TEST  RESTRAINTS  ADDED 
ALL  RESTRAINTS  REMOVED 


REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

Figure  9.  3-,  4-  and  6-Year  Programs  Test  Effectiveness 

Sensitivity  -  Ratio  of  Study  Value  and  Base¬ 
line  Value  Problem  Identification  Test  Costs 
for  Required  MTBR's. 
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4.4  TEST  OPERATING  TIME/CALENDAR  TIME  RELATIONSHIP 
Results 

The  test-programs  resulting  from  the  flight  test  versus  bench 
test  cost  optimization  for  each  test  operating  time/calendar 
time  study  parameter  are  shown  in  Appendix  I. 

The  3-,  4-,  and  6-year  Problem  Identification  Test  Costs  versus 
required  MTBR  achieved  plots  are  contained  in  Figures  25,  26, 
and  27,  in  Appendix  II. 

Figures  10,  11,  and  12  contain  the  plots  of  costs  relative  to 
the  associated  baseline  versus  required  MTBR  achieved  for  3-, 
4-,  and  6-year  programs,  respectively.  In  each  figure,  Case 
"A"  represents  the  baseline  case  evaluated  in  Volume  I. 

Absolute  values  of  problem  identification  test  costs  and  the 
cost  relative  to  the  associated  baseline  cost  for  each  test 
operating  time/calendar  time  case  are  summarized  in  Table  V, 
on  page  38.  Table  V  also  lists  the  baseline  case  absolute 
values  from  Volume  I . 

Discussion 

In  general,  the  problem  identification  test  costs  are  insensi¬ 
tive  to  changes  in  the  operating  time/calendar  time  relation¬ 
ship. 

Only  the  3-year  program  is  affected  when  the  flight  test  hours 
are  assumed  to  be  reduced  along  with  bench  test  hours.  For 
high  required  MTBR  achieved  values,  the  aircraft  production 
limit  is  reached  and  the  number  of  available  flight  hours 
becomes  fixed.  This  is  shown  in  Figure  10  by  curves  D  and  E. 

Fixing  the  available  flight  hours  forces  the  use  of  excessive 
amounts  of  bench  test,  reflecting  the  inefficiency  of  certain 
test  techniques.  In  the  case  of  the  25%  of  baseline  run  time, 
required  MTBR ' s  much  above  4,000  hours  cannot  be  reached  by 
increasing  bench  test  duration.  This  is  illustrated  in  curve 
E  of  Figure  10  by  the  dashed  portion  of  the  curve.  Curve  D 
(50%  x  OT/CT,  flight  test  included)  of  Figure  10  shows  a  rapid 
increase  in  the  relative  cost  as  the  value  of  required  MTBR 
achieved  is  increased.  In  this  instance  the  flight  hour  limit 
is  reached  but  the  MTBR  is  still  achieved  by  increased  levels 
of  bench  testing.  Extension  of  the  required  MTBR  much  past 
5200  hours  would  result  in  a  sharp  cost  ratio  increase  similar 
to  that  of  curve  E  (Figure  10)  . 

Except  as  discussed  above,  the  flight  limits  are  not  reached 
during  the  cost  optimization  process,  regardless  of  whether 
the  flight  hours  are  reduced  along  with  the  bench  test  hours. 

27 


J 


TEST  COST/BASELINE  TEST  COST 


28 


TEST  COST/BASELINE  TEST  COST 


6 


A  =  BASELINE 
B  =  OT/CT  .5 
C  =  OT/CT  .25 


REQUIRED  MTBR  ACHIEVED 

Figure  11.  4-Year  Program  Operating  Time  to  Calendar 

Time  Sensitivity  -  Ratio  of  Study  Value  and 
Baseline  Value  Problem  Identification  Test 
Costs  for  Required  MTBR's. 
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In  most  instances,  the  test  operating  time/calendar  time  var¬ 
iable  has  only  a  minor  effect  on  problem  identification  test 
costs.  The  one  exception  is  in  the  3-year,  25%  x  OT/CT  case. 
This  case  suffers  from  an  extremely  cramped  schedule.  The 
reverse  slope  of  curve  C  of  Figure  10  occurs  because  of 
scheduling  effects,  which  is  also  seen  in  Figure  25.  In  the 
baseline  case  the  higher  required  MTBR  values  were  already 
restrained  by  scheduling,  where  the  lower  lequired  MTBR  values 
did  not  encounter  restraints.  Escalation  of  the  low  MTBR 
values  to  a  schedule  restrained  condition  resulted  in  a  pro¬ 
portionally  greater  cost  increase. 

For  the  other  cases,  the  flight  optimization  process  compen¬ 
sates  for  the  decrease  in  test  run  time.  The  optimization 
process  permits  the  bench  test  hours  to  be  reduced  to  decrease 
the  requirement  for  test  rigs,  while  the  flight  test  hours  are 
increased  to  fill  out  the  testing  needed  to  give  the  required 
MTBR.  Generally,  for  the  near-optimum  values  of  the  cost 
optimization  curve,  the  bench  testing  is  somewhat  inefficient 
and  an  hour  of  flight  test  can  replace  several  hours  of  bench 
testing.  This  reduces  the  magnitude  of  any  cost  increase. 

4.5  SENSITIVITY  TO  COMBINED  VARIABLES 

General 


The  study,  until  now,  deals  only  with  cost  sensitivity  to 
single  variables.  This  section  of  the  study  identifies  the 
worst  case  evaluated  for  each  study  variable  and  then  combines 
these  individual  "worst  cases"  into  a  "combined  worst  case". 

The  degree  of  realism  for  each  case  of  each  study  variable  was 
tabulated  in  Section  2.0.  Table  IV  presents  a  summary  for  all 
of  the  study  variables.  The  individual  "worst  cases"  evaluated 
for  each  study  variable  are  identified  in  Table  IV. 

The  "combined  worst  case"  evaluated  represents  a  situation 
involving : 

2  x  Baseline  Failure  Mode  Quantity 

Maintenance  &  Test  Criteria  Artificial  Restraints  Present 

8  Observations  to  fix,  and  1/4  x  Baseline  Operating  Time/ 
Calendar  Time  Ratio. 

" Worst  Case"  Results 


The  test  programs  resulting  from  the  flight  test  versus  bench 
test  cost  optimization  for  the  "worst  case"  analysis  are  shown 
in  Appendix  I. 
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TABLE  IV.  SUMMARY  OF  DEGREES  OF  REALISM 

INDIVIDUAL 

CASES  STUDIED 

Degree  of 
Realism 

MTBR  Off- 
The-Board 

Test 

Effective¬ 

ness 

Corrective 

Action 

Efficiency 

OT/CT 

Optimistic , 

Improbable 

Case 

1/4  x 

Baseline 

Failure 

Mode 

Quantity 

No  Re¬ 
straints 

Baseline 
Efficiency 
(2  obs.  to 
fix) 

Baseline 

OT/CT  Ratios 

Expc  ted 
Case 

Baseline 

Failure 

Mode 

Quantity 

Baseline 

Restraints 

4  obs .  to 
fix 

1/2  x  Base¬ 
line  OT/CT 
Ratios 

Worst 

Case, 

Improbable 

2  x  Base¬ 
line 
Failure 
Mode 
Quantity 

Maint.  & 

Test 

Criteria 

Restraints 

Added  to 

Baseline 

Restraints 

8  obs .  to 
fix 

1/4  x  Base¬ 
line  OT/CT 
Ratios 

The  3-,  4- ,  and  6-year  problem  identification  individual  and 
combined  worst  case  test  program  cost  plots  versus  required 
MTBR  achieved  are  contained  in  Figures  28,  29,  and  30  of 
Appendix  II. 

Problem  identification  program  individ  mbined  worst 

case  test  costs  are  summarized  in  Tabl*  V. 

Figures  13,  14,  and  15  contain  plots  of  worst  case  costs 
relative  to  the  associated  baseline  costs  of  Volume  I  versus 
required  MTBR  achieved  for  3-,  4-  and  6-year  programs, 
respectively. 

"Worst  Case"  Discussion 


As  might  be  anticipated,  the  combined  worst  case  analysis  shows 
a  drastic  increase  in  test  program  cost.  Program  costs  es¬ 
calate  10  to  25  times  the  baseline  cost  of  Volume  I.  While  a 
life  cycle  cost  analysis  might  show  the  levels  of  reliability 
obtained  to  be  cost  effective  even  at  these  high  levels  of  test 
program  costs,  an  austere  cost  environment  may  nevertheless 
demand  that  development  program  costs  be  controlled  in  a 
direction  approaching  the  Volume  I  baseline  level. 
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REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

Figure  13.  3-Year  Program  Worst  Cast  Anal ’sis  -  Ratio 
of  Study  Value  and  Baseline  Value  Problem 
Identification  Test  Costs. 
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Significantly,  it  becomes  apparent  that  an  environment  must 
exist  that  will  cause  both  planning  and  implementation  to  oc¬ 
cur  in  the  i.reas  of  MTBR  of f-the-board ,  corrective  action 
efficiency,  test  effectiveness,  and  test  operating  time/ 
calendar  time  ratio  in  order  that: 

•  Necessary  MTBR's  will  be  achieved. 

•  Excessive  test  program  costs  will  be  avoided. 
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5 . 0  SUMMARY  OF  RESULTS 


Table  V  presents  a  summary  of  the  results  of  this  study 
Table  VI  presents  the  relative  effect  of  program  length 


on 
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TABLE  V.  SUMMARY  OF  STUDY  RESULTS 


Program  Length 

3  Years 

4  Years 

6  Years  l 

MTBR 

Re  1 . 

Rel . 

Rel. 

Caie 

Hours 

Million  $ 

Cost 

Mill  ion  $ 

Cost 

Million  S 

Cost 

600 

2.1 

1 

2.1 

1 

2.1 

1 

Base  line 

3000 

13.8 

1 

12.5 

1 

12 . 5 

1 

5200 

22.7 

1 

22.1 

1 

20.9 

1 

High-Failure- 

600 

2.1 

1 

2.1 

1 

2.1 

1 

Rate  Modes 

3000 

13.8 

1 

12  .  5 

1 

12.5 

1 

Removed 

5200 

22.7 

1 

22.1 

1 

20.9 

1 

Low-Failure- 

600 

2  .  1 

1 

2.1 

1 

2.  1 

1 

Rate  Modes 

3000 

9  .  7 

.  70 

9.4 

.75 

9.4 

.75 

Removed 

5200 

14.8 

.65 

14.0 

.63 

13.5 

.65 

All  Failure 

Modes  x  2 

600 

3000 

4.2 

28.0 

2 

2.03 

4.2 

27.2 

2 

2.18 

4 . 2 
26.1 

2 

2.09 

5200 

50.7 

2.23 

46.5 

2.10 

43.8 

2.10 

All  Failure 

600 

0 

0 

0 

0 

0 

0 

Modes  x  1/4 

3000 

3.4 

.25 

3.4 

.  27 

3.4 

.27 

5200 

6.0 

.26 

5.1 

.23 

5.2 

.24 

600 

5.7 

2.71 

5.4 

2.57 

5  .  4 

2.57 

4  Observations 

3000 

28.6 

2.07 

26.5 

2.12 

25.1 

2.01 

to  Fix 

5200 

53.0 

2.  34 

46.4 

2.10 

45.0 

2.15 

8  Observations 

600 

14.6 

6.95 

14.0 

6.67 

12.8 

6.10 

to  Fix 

3000 

63.4 

4.59 

57.9 

4.63 

54 . 4 

4.35 

5200 

a 

193 

8.51 

93 

4.21 

90 

4  .31 

Maintenance  and 

600 

2.6 

1.24 

2.6 

1.24 

2.6 

1.24 

Teat  Acceptance 

3000 

17.9 

1.30 

17.5 

1.40 

17.5 

1.40 

Criteria 
Restraints  Added 

5200 

27.5 

1.21 

26.4 

1.20 

26 . 2 

1.25 

All  Restraints 

600 

2.0 

.95 

2.0 

.95 

2.0 

.95 

Removed  (100% 

3000 

8.  3 

.60 

6 . 4 

.51 

6 . 4 

.51 

Ef  fectiveness ) 

5200 

11.8 

.  52 

10.6 

.48 

9  .  3 

.45 

.5  x  Opr.  Time/ 

6uu 

2  .  1 

1 

2  .  1 

1 

2  .  1 

1 

Cal.  Time  Except 

3000 

15.7 

1.14 

13.0 

1 .04 

12.5 

1 

Fit.  Hrs.  Not 

5200 

25.8 

1.14 

24.1 

1.09 

2  3.2 

1.11 

.25  x  Opr.  Time/ 

600 

3.4 

1.62 

2  .  1 

1 

2  .  1 

1 

Cal.  Time  Except 

3000 

18.5 

1.34 

16 . 2 

1  .  30 

13.9 

1.11 

Fit.  Hrs.  Not 
Reduced 

5200 

28.9 

1.27 

26  .  3 

1  .  19 

25.0 

1.20 

.5  X  Opr.  Time/ 

600 

2  .  1 

1 

2.1 

1 

2.1 

1 

Cal.  Time,  Incl. 

3000 

15.7 

1.14 

13.0 

1  .04 

12.5 

1 

Fit.  Hrs. 

5200 

b 

39.6 

1.74 

24.1 

1.09 

23.2 

1.11 

.25  X  Opr.  Time/ 

600 

3.4 

1.62 

2  .  1 

1 

2  .  1 

1 

Cal.  Time  Incl 

3000 

b 

27.6 

2.00 

16  .  2 

1 .  30 

13.9 

1.11 

Fit.  Hrs. 

5200 

c 

' 

26.3 

1.19 

25.0 

1.20 

Combined  Worst 

600 

48.6 

23.14 

40.  3 

19.20 

34  .  7 

16.52 

3000 

c 

— 

— 

144 

1  1  .  b2 

118 

9.44 

Case 

5200 

c 

' 

- 

305 

13.80 

264 

12.63 

NOTES : 


a.  Controls  Do  Not  Reach  Required  MTBR 

b.  Factory  Capacity  Limits  Flight  Hrs 

c.  MTBR  Not  Attainable 
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TABLE  VI.  RELATIVE  EFFECT  OF  PROGRAM  LENGTH  ON 
YEAR-TO-YEAR  PROGRAM  COST  VARIATION 


Program  Length 


Case 


Baseline 

High-Failure-Rate  Modes  Eliminated 

Low-Failure-Rate  Modes  Eliminated 

All  Failure  Modes  x  2 

All  Failure  Modes  x  1/4 

4  Observations  to  Fix 

8  Observations  to  Fix 

Maint.  &  Test  Accept.  Criteria 
Restraints  Added 

All  Restraints  Removed 

.5  OT/CT  Flight  Excluded 

.25  OT/CT  Flight  Excluded 

Worst  Case  (600  Hrs  Req’d  MTBR) 

Worst  Case  (3000  or  5200  Hrs  Req'd 
MTBR) 


-Year 

4 

-Year 

6 

1 

1 

1 

1 

1.11 

1.05 

1.29 

.98 

1.07 

1.05 

1.21 

1.21 


1 

1 

1 

.99 

1 

.99 

.95 

1.01 


6.0  CONCLUSIONS 


The  sensitivity  study  shows  that  the  numerical  reliability  re¬ 
quirement/reliability  test  cost  relationship  is  significantly 
affected  by  decisions,  philosophy,  and  approaches  taken  during 
the  development  program  with  respect  to  both  designing  for  re¬ 
liability  and  subsequent  reliability  testing. 

Contrary  to  initial  expectations,  however,  the  reliability/cost 
relationships  were  not  uniformly  sensitive  to  each  variable 
under  investigation.  Specifically,  the  following  was  found: 

•  The  test  effectiveness  and  operating  time/calendar 
time  variables  have  only  small  effects  on  the  problem 
identification  test  costs.  The  flight  versus  bench 
test  cost  optimizations  process  lets  increased  flight 
hours  overcome  inefficient  or  reduced  bench  test  at 
only  a  small  increase  in  cost. 

•  MTBR  of f-the-board  directly  affects  the  cost.  Where 
the  distribution  of  problems  is  equally  changed  across 
the  frequency  spectrum,  the  cost  increase  is  approxi¬ 
mately  equal  to  the  percentage  of  change  in  the  study 
variable.  Flight  test  efficiency,  in  these  instances 
is  affected  the  same  as  bench  testing. 

Design  reliability  analysis  and  support  activities  will 
reduce  test  costs  if  the  effort  is  directed  toward  re¬ 
ducing  low-  and  medium-frequency  failure  modes.  Current 
attention  given  to  high-frequency  problems  does  little 
to  reduce  the  test  durations  or  costs. 

•  Corrective  action  efficiency  is  a  critical  aspect  of 
problem  identification  costs.  Inefficiency  can  result 
not  only  in  increased  costs  but  also  in  failure  to 
achieve  the  required  MTBR  (e.g.,  50%  reduction  in  con¬ 
tractor  corrective  action  efficiency  can  result  in 
achieving  only  3000  hours  MTBR  when  5200  hours  is  re¬ 
quired,  regardless  of  the  amount  spent) .  The  achievement 
of  increased  efficiency  is  dependent,  primarily,  cn  the 
philosophies  and  controls  imposed  upon  the  test  and  cor¬ 
rective  action  procedures. 
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APPENDIX  I 

SUMMARY  OF  TEST  PROGRAMS 


Optimum  cost  test  programs  computed  for  this  study  are  sum¬ 
marized  in  Table  VII.  The  table  contains  the  test  rig  run 
times,  the  Type  II  flight  times,  the  number  of  test  rigs,  the 
test  rig  run  cost,  the  Type  II  flight  test  cost,  the  test  rig 
costs,  and  the  total  problem  identification  test  cost  for  each 
condition  investigated. 


Case 


Baseline 


High  failure  rate 
modes  removed 


Low  failure  rate 
modes  removed 


All  failure  modes 
X  2 


All  failure  modes 
X  1/4 


4  observations 
to  fix 


8  observations 
to  fix 


Maintenance  and 
test  acceptance 
criteria  added 

All  restraints 
removed  (100% 
effectiveness) 

.5  X  operating 
time/cal.  time 
except  fit.  hrs. 
not  reduced 

.5  X  operating 
time/cal.  time 
including  fit. 
hrs . 

.25  X  operating 
time/cal.  time 
excluding  fit. 
hrs . 

.25  X  operating 
time/cal.  time 
including  fit. 
hrs . 

Combined  worst 
case 


Type  I 
Fit. Test 
Time , 
Thous . 
Hrs 


Type  II 
Fit.  Test 


0 

0 

2.2 

5.5 

6.0 

15.0 

0 

0 

2.2 

5.5 

6.0 

15.0 

0 

0 

1.7 

4.2 

2.2 

5.5 

0 

0 

8.5 

21.2 

8.5 

21.2 

0 

0 

0 

0 

.7 

1.8 

.7 

1.8 

5.6 

14.0 

10.3 

25.8 

2.8 

7.0 

10.3 

25.8 

10.3 

25.8 

27.5 

68.8 

0 

0 

5.8 

14.5 

8.3 

20.8 

0 

0 

0 

0 

0 

0 

0 

0 

2.9 

7.2 

7.2 

18.0 

0 

0 

2.9 

7.2 

4.5 

1!  .2 

.6 

1.5 

4.9 

12.2 

CAN’T 

OBTAIN 

.6 

1.5 

1.5 

3.8 

CAN'T 

OBTAIN 

8.5 

21.2 

CAN'T 

OBTAIN 

CAN'T 

OBTAIN 

TABLE  VII. 


SUMMARY  OF  TEST  PROG 


3-YEAR  TEST  PROGRAM  SUMMARY 


Closed  Loop  Main 
XMSN  Test  Stand 


*  Fit.  controls  cannot  reach  required  MTBR  -  Fit.  hrs.  are  limited  at  factory  capacity. 
**  Restriction  of  factory  limit  deleted. 
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SUMMARY  OF  TEST  PROGRAMS 


3-YEAR  TEST  PROGRAM  SUMMARY 


V 

Ihirl  Tower 

Controls  Bench 

Test  Stand 

Ta i 1  Rotor 

Tost  Stand 

Total 
Run 
Cost , 
Mill. 
Doll. 

Total 
Acq  . 
Cost , 
Mill. 
Doll. 

Total 
Prog. 
Cost , 
Mill. 
Doll. 

Run 

Cost, 

Mill. 

Doll. 

No.  of 

Test 

Stands 

Stand 
Cost , 
Mill. 
Doll. 

Test 

Cost, 

Mill. 

Doll. 

Test 
Time, 
Thous . 
Mrs 

Run 
Cost , 
Mill. 
Doll. 

No .  of 

Test 

Stands 

Stand 
Cos  t , 
Mill. 
Doll. 

Test 
Cost , 
Mill. 
Doll. 

Tost 

T  i  mo , 
Thous . 
Iirs 

Pun 
Cost , 

Mill. 
Doll  . 

No.  of 

Test 

Stands 

Stand 
Cost , 
Mill. 
Doll  . 

Test 
Cost , 
Mill. 
Doll. 

0 

n 

0 

0 

.85 

.03 

1 

.  1 

.  1  3 

0 

0 

0 

0 

0 

.73 

1.4 

2.1 

.75 

i 

.56 

1.3 

7.2 

.29 

1 

.  1 

.  39 

2.8 

.  31 

i 

.  33 

1.0 

10.2 

3.6 

13.8 

.40 

i 

.56 

1.0 

1.4 

.06 

1 

.1 

.  16 

0 

0 

0 

0 

0 

19.5 

3.2 

22.7 

0 

0 

o 

0 

.85 

.03 

1 

.  1 

.  1  3 

n 

0 

0 

0 

0 

.73 

1.4 

2.1 

.75 

l 

.56 

1.31 

7.2 

.29 

1 

.  1 

.  39 

2.8 

.  M 

i 

.33 

1.0 

10.2 

3.6 

13.8 

.  4 

l 

.56 

.96 

1.4 

.06 

1 

.  1 

.16 

0 

0 

0 

0 

0 

19.5 

3.2 

22.7 

0 

0 

0 

0 

.6 

.02 

1 

.  1 

.13 

0 

1  0 

0 

0 

0 

.69 

1.4 

2.1 

.  6  2 

1 

.  56 

1.18 

3.4 

.  14 

1 

.  1 

.24 

1.25 

.  36 

1 

.33 

.69 

7.4 

2.3 

9.7 

1.1 

1 

.56 

1.66 

10.1 

.4 

1 

.1 

.5 

5.3 

.  58 

1 

.33 

.91 

11.2 

3.6 

14.8 

0 

0 

0 

0 

4.6 

.  18 

1 

.  1 

.28 

3.5 

.  38 

1 

.33 

.71 

2.4 

1.8 

4.2 

0 

0 

0 

0 

.  3 

.01 

1 

.1 

.11 

0 

0 

0 

0 

0 

25.3 

2.7 

28.0 

1.2 

1 

.  56 

1.76 

4.5 

.18 

1 

.1 

.28 

6.8 

.  75 

1 

.33 

1.08 

39.2 

11.5 

50.7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

n 

0 

0 

0 

n 

0 

0 

0 

0 

0 

0 

0 

2.7 

.11 

1 

.  1 

.21 

.8 

.on 

1 

.  33 

.42 

1.7 

1.7 

3.4 

0 

0 

0 

0 

4 

'  16 

1 

.  1 

.26 

2.9 

1 

.33 

.65 

4.2 

1.8 

6.0 

0 

n 

0 

0 

3.5 

.  14 

1 

.  1 

.24 

.5 

.06 

1 

.  33 

.  39 

4.6 

1.7 

5.7 

1.6 

2 

1.12 

1.7 

2.0 

.8 

2 

.2 

1  .0 

R .  fl 

.88 

1 

.33 

1.2 

28.0 

5.6 

28.6 

2.3 

*■> 

1.12 

3.4 

7.0 

.  28 

1 

.  1 

.  38 

7.2 

.79 

1 

.  33 

1.1 

45.0 

8.0 

53.0 

0 

n 

0 

0 

7.2 

.29 

1 

.  1 

.  39 

1.4 

.  15 

1 

.  3  3 

.48 

11.6 

3.0 

14.6 

3.7 

i 

3.7 

7.4 

80.0 

1.4 

5 

.5 

3.1 

18.  1 

2.0 

3 

.99 

3.0 

51.5 

15.6 

67.1 

8.5 

14.1 

22.6 

21.0 

.84 

2 

O 

1.0 

72.2 

0 

1.0 

10.9 

125 

68 

193* 

2.5 

2 

l.  12 

3.6 

6.0 

.24 

1 

.  1 

.  34 

5.2 

r  7 

1 

.  13 

.9 

88.1 

11.9 

100** 

0 

0 

0 

0 

1.0 

.04 

1 

.  1 

.14 

n 

n 

n 

n 

0 

1.2 

1.4 

2.6 

0 

0 

0 

0 

.  3 

.01 

1 

.  1 

.  11 

0 

n 

n 

n 

0 

16.5 

1.4 

17.9 

0 

0 

0 

0 

.6 

.02 

1 

.  1 

.12 

0 

n 

n 

0 

0 

24 .  R 

2.7 

27.5 

0 

0 

0 

0 

1.0 

.038 

1 

.  1 

.14 

n 

0 

0 

0 

0 

.6 

1.4 

2.0 

1.1 

2 

1.  12 

1.2 

2.5 

.  1 

1 

.  1 

.2 

3.5 

.  30 

i 

.  33 

.7 

4.2 

4.1 

R.  3 

1.65 

2 

1.  12 

1.8 

0 

0 

0 

0 

0 

5.5 

.60  6 

i 

.33 

.9 

6.5 

5.3 

11.8 

0 

0 

0 

0 

.85 

.03 

1 

.  1 

.  1  3 

0 

0 

0 

n 

0 

.73 

1.4 

2.1 

.62 

1 

.56 

1.2 

2.0 

.08 

1 

.  1 

.18 

2.5 

.  2R 

i 

.  n  i 

.6 

10.8 

4.9 

15.7 

.13 

1 

.56 

.7 

1.4 

.06 

1 

.  1 

.  16 

0 

0 

0 

n 

0 

21.2 

4.6 

25. R 

0 

0 

0 

0 

.85 

.03 

1 

.  1 

.  1  3 

0 

n 

0 

n 

0 

.73 

1.4 

2.1 

.62 

1 

.  56 

1.2 

2.0 

.08 

1 

.  1 

.18 

2 .  c 

.  28 

1 

.  33 

.6 

10.8 

4.9 

15.7 

.95 

2 

1.12 

2.1 

3.0 

.12 

1 

.  1 

.22 

3.1 

.  34 

1 

.33 

.67 

23.7 

15.9 

31.6 

0 

0 

0 

0 

.1 

0 

1 

.  1 

.  1 

0 

0 

0 

0 

0 

2.0 

1.4 

3.4 

1.0 

1 

.  56 

1.6 

.04 

0 

1 

.  1 

.  1 

n 

0 

n 

n 

0 

13.9 

4.6 

18.5 

0 

n 

0 

0 

.  1 

0 

1 

.  1 

.  1 

n 

n 

n 

0 

0 

2.0 

1.4 

3.4 

.97 

4 

6.  3 

7.3 

13.7 

.  54 

4 

.4 

.9 

4.9 

.  c4 

i 

.  09 

1  .5 

9.5 

18.1 

27.6 

0 

0 

n 

0 

12.1 

.48 

4 

.  4 

12.5 

5.5 

.6  1 

3 

.09 

1.6 

1 

j _ 

29.0 

17.6 

4  8.6 

TABLE  VII  -  (Cont. ) 


4-VI'AI-  TI.K’I  I'Wifil-AM  r,i  MMARY 


1  Type  1 1 

Type  I 

lit 

.  Tost 

\Msr: 

Tent  St. 

ind 

Wh  l t  1  Towr  r 

1*  1 1 .  I'es  t 

Test 

Hull 

St  and 

Test 

Test 

Pun 

Stand 

Ten  t 

Ti  s  t 

Keq ' d  . 

Time , 

Time , 

Cos  t , 

Tin*' , 

Tost  , 

No.  ot 

Cost  , 

Cost  , 

1*  i  ITU  , 

Cost  , 

*:<•>.  of 

Cn.it, 

Con  L  , 

T  i  me , 

MTHK 

Thous . 

Thous 

.  'Ml], 

Thous ♦ 

'1  i  1  1  . 

Test 

'1  i  1  1  . 

*ii  1 1 . 

Thous . 

Mill. 

T«  s  t 

.••■.in. 

M  ill. 

Thous . 

Hrs 

Mrs 

Hrs 

liol  .  . 

Hrs 

Ool  1  . 

Stands 

Pol  1 . 

Poll. 

Mrs 

Poll. 

!  t  ir.dv 

lol  1  . 

Pol  1  . 

Mrs 

line 

MM 

1.5 

0 

2.5 

7 

1 

1  .  3 

2.0 

0 

0 

0 

0 

0 

.85 

111 

1.5 

2  2 

5.5 

12 

1.  36 

1 

1  .  3 

4.7 

3.4 

.  7  5 

1 

.  56 

1  .  3 

: .  2 

II 

1  .  5 

4.5 

11.2 

22.0 

6.05 

2 

2.6 

8.6 

4.2 

.9? 

1 

.  00 

1 .  5 

2.15 

failure  rate 

600 

1  .  5 

0 

0 

2.  35 

.  ()(» 

1 

1  .  3 

2.0 

0 

0 

n 

0 

.  8 

es  removed 

3000 

1 . 5 

2  •  2 

5.5 

12 

3 .  36 

1 

1.  3 

4.7 

3.4 

7.5 

i 

.  00 

1.3 

7.2 

5200 

1 . 5 

4.5 

11.2 

2  2.6 

6.05 

2 

2.6 

a.o 

4.2 

.97 

i 

.  56 

1.5 

2.15 

failure  rate 

600 

1.5 

0 

0 

2.7 

.  7 

1 

1  .  3 

2.0 

0 

0 

0 

o 

EH 

es  removed 

3000 

1  .  5 

1.5 

3.8 

7.6 

2.2 

1 

1 .  3 

3.5 

3.0 

.  m 

i 

.56 

1.2? 

* 

4.3 

5200 

1.5 

2.7 

6.8 

11.9 

3.3 

1 

1.3 

4.7 

4.6 

1.0 

i 

.56 

1.5 

5.0 

failure  modes 

600 

1.0 

0 

0 

6.8 

1.9 

1 

1 .  3 

3.2 

0 

0 

0 

0 

4 . 6 

3000 

1 . 5 

7.3 

18.2 

19.5 

5.5 

2 

2.6 

8.1 

1  .4 

.31 

1 

.56 

.  87 

.  7 

5200 

1  .  5 

7.9 

19.8 

6  0 

16.8 

5 

6.0 

23.3 

6.  1 

1.3 

1 

.56 

1  .  86 

4 . 9 

failure  modes 

600 

1 . 5 

0 

0 

0 

0 

0 

0 

0 

n 

0 

0 

0 

o 

o 

f  4 

3000 

1  .  5 

0 

0 

5.2 

1  .  5 

1 

1  .  3 

2.8 

0 

0 

0 

0 

0 

2.7 

5200 

1  .  0 

0 

0 

9.7 

2.7 

1 

1  .  3 

4.0 

0 

0 

0 

0 

0 

5.6 

lervat ions 

mo 

1  .5 

.5 

1.2 

7.8 

2.2 

1 

1  .  3 

3.5 

0 

0 

0 

o 

3. 9 

fix 

30U0 

1.5 

5.5 

13.8 

20.8 

r> .  8 

? 

?.f, 

8.4 

7.7 

1  .7 

1 

.  56 

O  7 

21 

5200 

1.5 

11.5 

28.8 

36.0 

10.1 

1 

3.n 

34.0 

a.7 

1.9 

1 

.56 

]  .  5 

4.0 

lervat ions 

6  00 

I  .5 

1.6 

l.o 

11.7 

\ 

1 

1.3 

4.6 

n 

0 

{> 

0 

0 

5 . 8 

fix 

3000 

1  .  5 

1M  .  5 

Jf- .  2 

51.5 

1  4  .  4 

5 

0.6 

20.9 

16.8 

X .  7 

2 

1.12 

4 . 8 

74.0 

5200* 

1.5 

2 .  7 

50 .  L 

7  5 . 5 

21.1 

0 

7  .H 

28.9 

1  8  .  ? 

4.o 

1.12 

5.1 

13.0 

tenance  and 

mo 

1  .  5 

0 

0 

4  . 

1  .  2 

1 

i . ; 

2.5 

n 

n 

0 

o 

1 .  n 

t  acceptance 

3oro 

1  .  5 

5  •  1 

1  -..2 

10 

3.5 

i 

1  .  3 

4  .  1 

n 

0 

n 

o 

o 

1 .  o 

teria  added 

5200 

1  .  5 

..’I  .  R 

12.0 

t .  4 

l 

1  .  3 

4  .  7 

0 

0 

n 

0 

0 

0 

restraints 

600 

1.5 

(i 

0 

2.0 

.  r'« 

l 

1  .  3 

1  .9 

n 

0 

o 

o 

o 

1 . 0 

>ved  (100?. 

3000 

1 . 5 

(l 

0 

9.0 

2.6 

i 

1  .  3 

3.  8 

s  .n 

1.1 

i 

r  6 

*>  t 

ictiveness ) 

5200 

1.5 

0 

14.0 

1.9 

2 

2.0 

6.5 

7.5 

i 

.  56 

2 . 2 

0 

operating 

600 

1  .5 

0 

0 

2.5 

.  7 

1 

1  .  ' 

7 .  n 

n 

0 

0 

o 

o 

.85 

l/cal.  tina 

3000 

1  .  5 

5.0 

11.7 

3.  3 

2 

2.6 

q ,  <i 

3.9 

.Pi 

i 

.  Ob 

1.42 

5.  1 

ipt  fit.  hrs. 

5200 

1 .  5 

6.9 

17.2 

12.0 

\ . 

2 

2.6 

6 .  m 

0 

1 

.  06 

1 .  0 

reduced 

operating 

600 

1.5 

n 

2.5 

.  7 

i 

1  .  3 

2.0 

n 

n 

n 

0 

0 

.  85 

l/cal.  time 

3000 

1  .  5 

2.0 

5.0 

11.7 

3.  3 

2.6 

5.9 

3.9 

.  pc 

1 

.  06 

1.42 

0.  1 

luding  fit. 

5200 

1.5 

6.9 

17.2 

12.0 

3.  36 

SI 

2.6 

5.r  0 

.  q 

*> 

, 

.56 

.  76 

1 . 5 

;  operating 

600 

1  .  5 

0 

2.5 

.  7 

mi 

1  .  3 

2.0 

n 

n 

n 

0 

0 

.  85 

l/cal.  time 

3000 

1.5 

3.2 

8.0 

8.7 

2.4 

3 

3.9 

6.  1 

2.4 

.  54 

} 

.  06 

1  .  1 

1  .  8 

.tiding  fit. 

5200 

1.5 

7.0 

19 . 8 

9.0 

2.5 

3.9 

o .  A 

0 

0 

n 

0 

0 

1  .  1 

operating 

600 

1  .  5 

0 

2.5 

.  7 

1 

1  .  3 

2.0 

0 

0 

0 

0 

0 

.  85 

l/cal.  time 

3000 

1.5  1 

3.2 

8.0 

8.7 

2.4 

3 

3.9 

0 .  3 

2.4 

.  5  4 

1 

.56 

1 . 1 

1  .  S 

uding  fit. 

5200 

1.5 

7.9 

19.8 

9 . 0 

2.5 

3 

3.9 

6.4 

0 

n 

n 

0 

n 

1  .  1 

ned  worst 

mo 

1.5 

(3.5 

21.2 

23.5 

6  .  6 

8 

10.4 

17.6 

0 

0 

n 

0 

n 

1  3.  1 

3000 

1  .  5 

43.5 

108.2 

4.  H 

1  3 

17 

22.  1 

35.  1 

n 

0 

0 

1) 

0 

0 

5200 

1  .5 

58.5 

146.2 

5 

60 

72 

97 

3  57 

o 

1  .  3 
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?t  r 

} 

'It.  controls  cannot  reach  required  MTBR  -  Kit.  hrs.  are  limited  at  factory  rapacity, 
lestriction  of  factory  limit  deleted. 
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Case 

Kt'q  * **  d . 

MTHN 

Mrs 

TV!’*'  I 
Fit.  1'esl 
T  i  me , 

Tin  ms. 
Hrs 

Tv 
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Tinu-, 

Tlxms 

Hrs 

Baseline 

600 

1.5 

0 

la 

1  .  5 

1.5 

1  ) 

4.5 

High  failure  rate 

600 

1 . 5 

0 

modes  removed 

3000 

l  .  5 

2.2 

5200 

1.5 

4.5 

Low  failure  rate 

600 

1.5 

0 

modes  removed 

3000 

1  .5 

1.5 
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APPENDIX  II 

PROBLEM  IDENTIFICATION  TEST  COSTS 


Problem  identification  test  costs  vs.  required  MTBR  are  shown 
in  Figures  16  through  30  for  3-,  4-,  and  6-year  programs,  for 
each  case  of  each  study  variable. 
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A  =  BASELINE  (2  OBSERVATIONS  TO  FIX) 
B  =  4  OBSERVATIONS  TO  FIX  |.  I 

C  =  8  OBSERVATIONS  TO  FIX  I  I 


REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

Figure  19.  3-Year  Program  Corrective  Action  Efficiency 

Sensitivity  -  Problem  Identification  Test  Costs 
for  Required  MTBR's. 


PROBLEM  IDENTIFICATION  TEST  COSTS  ($  MILLIONS) 


PROBLEM  IDENTIFICATION  TEST  COSTS  ($  MILLIONS) 


C  1  '  2  3  4  5 

REQUIRED  MTBR  ACHIEVED  (10  00  HRS) 

Figure  21.  6-Year  Program  Corrective  Action  Efficiency 

Sensitivity  -  Problem  Identif ication  Test  Costs 
for  Required  MTBR's. 
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PROBLEM  IDENTIFICATION  TEST  COST  ($  MILLIONS) 


PROBLEM  IDENTIFICATION  TEST  COST  ($  MILLIONS) 
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PROBLEM  IDENTIFICATION  TEST  COST  ($  MILLIONS) 


ILLION 


Figure  28.  3-Year  Program  Worst  Case  Analysis  -  Problem 
Identification  Test  Cost  for  Required  MTBR's 


REQUIRED  MTBR  ACHIEVED  (1000  HRS) 

4-Year  Program  Worst  Case  Analysis  -  Problem 
Identification  Test  Cost  for  Required  WTBR's 


lTION  test  COST  ($ 


REQUIRED  MTBR  ACHIEVED  (1000  HRS) 


Figure  30.  6-Year  Program  Worst  Case  Analysis  -  Problem 
Identification  list  Cost  for  Required  MTBR's. 
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